On Techniques to Improve Robustness and Scalability of a Parallel Hybrid Linear Solver

A hybrid linear solver based on the Schur complement method has great potential to be a general purpose solver scalable on tens of thousands of processors. For this, it is imperative to exploit two levels of parallelism; namely, solving independent subdomains in parallel and using multiple processors per subdomain. This hierarchical parallelism can lead to a scalable implementation which maintains numerical stability at the same time. In this framework, load imbalance and excessive communication, which can lead to performance bottlenecks, occur at two levels: in an intra-processor group assigned to the same subdomain and among inter-processor groups assigned to different subdomains. We developed several techniques to address these issues, such as taking advantage of the sparsity of right-hand-sides during the triangular solutions with interfaces, load balancing sparse matrix-matrix multiplication to form update matrices, and designing an effective asynchronous point-to-point communication of the update matrices. We present numerical results to demonstrate that with the help of these techniques, our hybrid solver can efficiently solve large-scale highly-indefinite linear systems on thousands of processors. 1 The Schur Complement Method and Parallelization Modern numerical simulations give rise to large-scale sparse linear systems of equations that are difficult to solve using standard techniques. Matrices that can be directly factorized are limited in size due to large memory requirements. Preconditioned iterative solvers require less memory, but often suffer from slow convergence. To mitigate these problems, several parallel hybrid solvers have been developed based on a non-overlapping domain decomposition idea called the Schur complement method [5,7]. In the Schur complement method, the original linear system is first reordered into a 2× 2 block system of the following form: ( A11 A12 A21 A22 )( x1 x2 ) = ( b1 b2 )